1. Field of the Invention
The invention relates to the evaluation of clinical trial protocols, and more particularly to a methodology and structures for assisting reviewers in the identification of operational uncertainties in clinical trial protocols.
2. References
The following documents are incorporated by reference herein:    DataEdge LLC. Indexes of clinical study complexity, 1993-1999. In: Mathieu M P, editor. Parexel's Pharmaceutical R&D Statistical Sourcebook 2000. Waltham, Mass.: Parexel International Corp; 2000. p. 66.    DataEdge LLC. Indexes of clinical trial costs per patient, 1993-1999. In: Mathieu M P, editor. Parexel's Pharmaceutical R&D Statistical Sourcebook 2000. Waltham, Mass.: Parexel International Corp; 2000. p. 67.    Grosso W E, et. al., “Knowledge Modeling at the Millennium (The Design and Evolution of Protégé-2000),” SMI Report Number: SMI-1999-0801 (1999), available at http://smi-web.stanford.edu/pubs/SMI_Abstracts/SMI-1999-0801.html, visited Jan. 1, 2000.    Holford N H, Kimko H C, Monteleone J P, Peck C C. Simulation of clinical trials. Annu Rev Pharmacol Toxicol 2000; 40:209-34.    Howard R A, Matheson J E, editors. Readings on the principles and applications of decision analysis. Menlo Park, Calif.: Strategic Decision Group; 1983.    Hulley S B, Cummings S R, Browner W S, Grady D, Hearst N, Newman T B. Designing clinical research: An epidemiologic approach. Second ed. Philadelphia: Lippincott Williams & Wilkins; 2001.    International Conference on Harmonisation. E6: Guideline for good clinical practice. URL: www.ifphma.org/ich5e.html. Accessed: 31 Aug. 2001.    International Conference on Harmonisation. E8: General considerations for clinical trials. URL: www.ifpma.org/ich5e.html. Accessed: 31 Aug. 2001.    Kroll J A, de Bruin A, Getz K, Eschmann K, Zisson S. Study conduct delays are getting worse. In: Kroll JA, editor. An Industry In Evolution. Second ed. Boston, Mass.: CenterWatch; 1999. p. 99.    Matheson D, Matheson J E. The smart organization: Creating value through strategic R&D. Boston, Mass.: Harvard Business School Press; 1998.    McNamee P, Celona J. Decision analysis for the professional with Supertree: Scientific Press; 1987.    Musen M A, Gennari J H, Eriksson H, Tu S W, Puerta A R. Protege-II: Computer support for development of intelligent systems from libraries of components. Medinfo 1995; 8(1):766-70.    Musen M A, Rohn J A, Fagan L M, & Shortliffe E H, “Knowledge engineering for a clinical trial advice system: Uncovering errors in protocol specification,” Report KSL-85-51, Proceedings AAMSI Congress 1986 (Levy, A. H. and Williams, B. T., eds.), pp. 24-27, Anaheim Calif., May 1986.    Musen M A. Domain ontologies in software engineering: Use of Protege with the EON architecture. Methods Inf Med 1998; 37(4-5):540-50.    Piantadosi S. Clinical trials: A methodologic perspective. New York, N.Y.: John Wiley & Sons; 1997.    Sheiner L B, Steimer J L. Pharmacokinetic/pharmacodynamic modeling in drug development. Annu Rev Pharmacol Toxicol 2000; 40:67-95.    Spilker B. Guide to Clinical Studies and Developing Protocols. New York, N.Y.: Raven Press; 1984.    Spilker B. Guide to Clinical Trials. New York, N.Y.: Raven Press; 1991.    Wampler S. Tackling Protocol Complexity. Good Clinical Practice J 2000; 7(2):6-8.
3. Description of Related Art
Clinical trial protocols are designed to produce scientifically sound new knowledge about the safety, efficacy or specific therapeutic characteristics of a new drug or treatment combination. Clinical trial protocols developed using modern study design principles have well-understood statistical hypothesis testing and internal validity characteristics. Thus, modern clinical studies are designed to deliver the most supportable scientific knowledge with the smallest number of study subjects. Despite these more efficient study designs, the complexity and associated costs of executing new protocols continue to increase.
The rising complexity of current protocol designs has resulted in a dramatic increase in the operational management overhead required to initiate, execute, and complete a clinical trial successfully within budget and time frame. With the increasing number of patients, investigators, locations, and countries involved in a given trial, it is not surprising that a many clinical trials have significant operational issues that cause substantial cost/time over-runs or outright trial failures.
A distinction between scientific versus operational issues in clinical trials planning and execution is important. A scientific issue arises due to the limited state of current knowledge about the trial agent(s) pharmacologic and therapeutic properties in the experimental clinical situation. This lack of scientific knowledge is precisely the reason a well-designed clinical trial is required. Scientific issues are ethically justified and do not indicate any problem with the clinical trial protocol. Operational issues arise because of unforeseen difficulties in executing the trial within the strict parameters or assumptions embedded implicitly or explicitly within the protocol design. In this case, the protocol designers may not have been able to predict the difficulties the field organization or clinical investigators may have in operationalizing specific study design components.
Operational deficiencies can be costly to an organization, especially if they require an amendment to the protocol during execution of the trial. Even simple amendments have substantial costs, including the internal overhead of detecting the need for an amendment, costs for creating the amendment, internal approvals costs for releasing the amendment, costs for disseminating the amendment and the follow-up effort to ensure that the amendment has been incorporated into the clinical trials process at a potentially large number of the trial sites. The cost of an amendment also includes the opportunity costs due to study completion delays or data analysis impact due to inconsistent operational behavior at the trial sites.
Amendments are only the most visible and more costly manifestation of a spectrum of ways in which operational deficiencies can impact an organization. A large number of operational issues are handled via formal and informal communications between study project managers and clinical trials sites. Thus the volume of faxes and telephone calls can be another “cost” which directly affects team productivity and trial site performance but is not as visible as a trial amendment.
At most large clinical development organizations, concern about protocol design quality has spawned the creation of Protocol Review Committees (PRCs). Although specifics vary widely, the role of the PRC is to provide a centralized resource for the critical examination of a proposed protocol in a “near-final” form. Unlike Institutional Review Boards who are concerned about patient safety, risk/benefit and informed consent, the focus of the PRC is on scientific merit, study validity, and the appropriate use of scarce clinical development resources. Many PRCs include representatives from clinical operations and a few include clinical trial sites so that the operational issues that may be embedded within or implied by a proposed trial design can be examined.
Many organizations that have implemented PRCs now require every protocol to be reviewed and approved by the committee prior to its release to the clinical operations team for field deployment. In addition, many PRC members state that a review by the committee often results in substantial changes to the original protocol. Protocol Review Committees bring together substantial skills and corporate institutional experience, representing an enormous investment of highly trained personnel. For the organizations that have committed to this approach, this investment is deemed to be justifiable, given the high stakes and resources committed to the execution and success of each trial.
PRC reviews do seem to have a positive contribution to the operational quality of reviewed protocols. But the cost of such reviews is large. PRCs generally require highly trained, expensive senior people to perform time-consuming, detailed review of every protocol prior to internal approval. The man-hours consumed in analyzing a proposed protocol, discussing the findings at a PRC meeting, presenting the findings to the protocol author and then repeating the process in a limited manner after the protocol is revised represent a huge hidden cost. As PRC members leave or rotate, the quality and quantity of protocols reviewed by the PRC may vary widely. Thus, while PRC reviews may be effective, they are neither scalable nor repeatable.
In addition to or instead of forming Protocol Review Committees, many organizations have instituted other methods for improving the quality of the protocol during its initial creation. Templates, checklists, and previously approved protocols are common materials provided to protocol authors to assist with the improvement of the quality of their initial protocol designs. These methods have numerous drawbacks, however, that significantly limit their usefulness. First, most paper-based methods are static. That is, these methods are not easily updated, disseminated, and then incorporated into the protocol writer's daily routine. Thus, it is often the case that protocols developed with these tools continue to make the same operational mistakes long after the organization has updated the reference materials.
Second, as people leave the organization, the institutional knowledge of what makes a protocol “work” within that organization is lost. If the employee was a member of the Protocol Review Committee, this loss of institutional knowledge is even more extensive, impacting the entire range of protocols reviewed by the PRC.
Third, organizational mergers and alliances result in widely disparate approaches, assumptions, and standard operating procedures for designing protocols. Operational knowledge unique to one organization or to a specific therapeutic area tends to remain within the original organization or therapeutic area and therefore not benefit the combined organization. Hard-earned (and expensive) experience-based knowledge diffusion occurs only if and when people from one organization migrate into similar positions within the second organization. Of course, this migration results in the loss of operational knowledge from the original clinical operations group.
Accordingly, there is an urgent need for a methodology and tools that will assist the reviewers of clinical trial protocols in identifying operational uncertainties early, before a clinical trial begins according to the protocol.